A single neuron network, which may be configured in a single layer is known as a perceptron. In general, a perceptron accepts multiple inputs, multiplies each by a weight, sums the weighted inputs, subtracts a threshold, and limits the resulting signal, such as by passing it through a hard limiting nonlinearity. A perceptron is described in W. McCulloch and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity", Bulletin of Mathematical Biophysics, Vol. 5, pp. 115-133, 1943. This perceptron was not itself useful until combined into multiple perceptron networks known as artificial neural networks. Such networks generally comprised interconnected perceptron layers having multiple input, intermediate and output neurons. For an example of such networks, see the text by J. Anderson and E. Rosenfeld, Neurocomputing: Foundations of Research, the MIT Press, 1988. Artificial neural networks currently are used in such applications as pattern recognition and classification.
While it is known to perform linearly separable functions using single layer neural networks, one shortcoming of such networks is that they heretofore have been thought to be inherently incapable of performing non-linearly separable functions. Specifically, it has been established that a two-input, single-layer, single-neuron network cannot implement such non-linearly separable functions as the exclusive-or ("XOR") and exclusive-nor functions ("XNOR"). See R. Hecht-Nielsen, Neurocomputing, 17-18, Addison-Wesley, 1989.
The rationale for the belief that single-layer neural networks incapable of performing non-linearly separable functions can be seen by examining the characteristics of a conventional neural network. Such a conventional neural network having two inputs, x.sub.1 and x.sub.2, can be represented by a hyperplane of w.sub.1 x.sub.1 +w.sub.2 x.sub.2 -.theta.=0, where w.sub.1 and w.sub.2 are weights applied to inputs x.sub.1 and x.sub.2 before application to the neuron, and .theta. is a threshold. The decision region in the x.sub.1 -x.sub.2 plane, i.e., the region containing input coordinates which yield a common output, is only a half-plane bounded by the hyperplane. A logical function is said to be linearly separable if it conforms to the hyperplane criterion, i.e., all input pairs having coordinates on one side of the hyperplane yield one result and all inputs with coordinates on the other side yield a different result. Because there is no single straight line subdividing the x.sub.1 -x.sub.2 plane, such that the inputs (0,0) and (1,1) will fall on one side of the hyperplane and thus yield a logical zero, and (0,1) and (1,0) on the other to yield a logical one, it was supposed that no combination of values for w.sub.1 and w.sub.2 will produce the input/output relationship of the XOR function for a single layer neural network. That is to say, the XOR function is a non-linearly separable function.
FIG. 11 shows the geometry of the plane/hyperplane discussed above. It can be seen that hyperplane 1 is incapable of separating the x.sub.1 -x.sub.2 plane into a pair of half planes having (0,1) and (1,0) (which yields an XOR result of 1) on one side of the hyperplane, and (0,0) and (1,1) (which yields an XOR result of 0) on the other.
The XOR operation is a common one in neural networks. To implement it, however, has required the use of multiple layers of neural networks. To reduce the number of components and hence the cost of networks, a need has arisen for a single layer neural network which can be trained to perform the XOR operation, as well as other non-linearly separable and separable operations, on one or more input signals.